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Smartphones/mobile devices are enduring all the aspects of human life. With 
the significant increase in demand for applications running on 
smartphones/mobile devices, developers and testers are anticipated to deliver 
high quality, on time and within budget applications. The estimation of 
development and testing provides a baseline and act as a tracking gear for 


stakeholders and developers. There are various approaches for estimation of 





traditional software development. But mobile applications are considered 
different from traditional software such as those running on desktop, laptop 
or on the web. Many traditional estimation techniques used for this software 
are adapted to mobile domain. With agile software development (ASD) 
methodology, the scenario of development and estimation has changed 
drastically and so as mobile app development and estimation. This paper 
provides a Systematic Literature Review (SLR) on traditional estimation 
techniques and agile estimation techniques applied in mobile 
software/application. Also, effort attributes and accuracy parameters for 
estimation in mobile apps are presented. However, to date, there are very 
fewer studies done on the mobile application estimation domain using agile 
methodology. 
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1. INTRODUCTION 

Background: - Because of popularity and ease of use of mobile devices has become the most 
indispensable expedient for human essentials from past few years [1]. Mobile software developers’ are driven 
to release software on time and within budget. For this purpose software estimation plays a very pivotal role 
in providing the most accurate sizing figure for building confidence in developers and stakeholders 
relationship [2]. Accuracy in estimation is a crucial factor for planning software development to evade budget 
overruns and tardy delivery of software. Estimation of test effort along with development is considered 
crucial for apprehending quality software [3]. Development of mobile software is considered different from 
developing traditional software pertaining to its distinctive features such as limited memory, processing 
power, small and multiple input interface, multiple connections, bandwidth factor, a lower battery, etc. [4-7]. 

With the advent of Agile Software Development (ASD) [8-9], entire software development 
community has been driven to the adoption of agile methodology. The Agile espousal to mobile application 
development is considered as a natural fit by many researchers [10-14]. The effort estimation of software 
development in an agile environment is also different from traditional software estimation [8]. Effort 
estimation is more exigent in an agile because of employing dynamic changes in requirements [15]. There are 
many estimation techniques for traditional software estimation in an agile environment such as planning 
poker, Expert judgment, Use Case Points (UCP) Method, etc. [8, 16]. 
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Mobile applications are different from traditional software [4-7] and agile approach to mobile 
application development and estimation needs a solid methodological approach for predicting effort. 

The Problem: - The approaches used for estimation of traditional software are adapted for mobile 
application development. But presently, not much work has been dedicated to identifying suitable approaches 
exclusively for effort estimation of mobile apps [17]. The estimation techniques in agile software 
development for traditional software cannot be adapted to mobile domain pertaining to different 
characteristics of mobile apps. However, there is very less literature available on the estimation of mobile 
apps in an agile environment 

Proposed solution: - This paper contributes mainly by examining the state-of-art of techniques 
applied for estimation of mobile software/applications in traditional software development and agile software 
development by using Systematic Literature Review (SLR). SLR will form baselines for mobile app 
developers for the selection of appropriate estimation method according to their need. This will also help 
researchers in filling the gap by proposing formal estimation models for mobile apps considering its specific 
characteristics. 

The rest of this paper is ordered as follows: Section 2 presents the Research method used for 
conducting the Systematic Literature Review (SLR). Section 3 presents the results of SLR and some 
discussions on selected studies and finally, Section 4 concludes the paper with future directions. 


2. RESEARCH METHOD 

The goal of this study is to gain an understanding of the current state-of-art in mobile app 
development effort estimation. To this end, an explorative study is conducted using the Systematic Literature 
Review (SLR) succeeding the guidelines postulated by Kitchenhan and Charters [18]. SLR is a research 
method for carrying out a literature review in a systematic way of charting well-defined phases. SLR method 
uses three phases for performing literature review including Planning and specifying research questions as a 
first phase, the second phase is conducting the review that comprises an identification of search string & data 
sources, selecting studies & data extraction and the third one being results reporting. 


2.1. Planning Phase 
For the smooth conduct of systematic literature review, proper planning is necessary. The research 
questions derive from the entire systematic literature review planning phase. 


2.1.1. Research Questions (Rqs) 

Affirming the research questions is the vital part of any systematic review. In accordance with 
guidelines proposed by Petticrew and Roberts [19], the criteria to frame research questions are based on 
PICOC (Population, Intervention, Comparison, Outcomes, and Context). If the research question is not 
outlined properly, the literature review may turn out off the course. For this study PICOC are defined as:- 


Population: Mobile Application projects 

Intervention: Effort estimation techniques/methods/process 

Comparison: Traditional mobile app effort estimation techniques with an agile mobile app estimation. 
Outcomes: Estimation models to follow for mobile app development in agile and traditional software 
development. 

Context: Review the existing studies on estimation of mobile apps. 

The research questions steer the entire systematic review methodology. The major objective of this finding is 
to answer the following research question:- 

RQ1. What are the traditional effort estimation techniques used for mobile software/application development 
and testing? 

RQ2. What is currently known effort estimation techniques followed in agile mobile software/application 
development and testing? 

RQ3. What are the estimation attributes and accuracy parameters used in estimation techniques for mobile 
application? 


2.2. Conducting the Review Phase 
2.2.1 Search Strategy 

The aim of directing search strategy is to identify studies pertaining to answer the RQs. Further, 
search strategy can be conducted in three phases; Identifying keywords and Defining Search strings, 
Identifying the data sources and Search Process in Data sources. 
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a) Identifying keywords and Defining Search Strings 

The first phase comprises forming the search string. The search strategy is set up to describe search 
strings and primary data sources. The guidelines provided by [18] are followed to define the search string by 
analyzing the main keywords in RQs, synonyms of the keywords and on any other spellings of the words. 
The identified keywords are shown in Table 1. 


Table 1. Identified Keywords and synonymous 








Keywords Synonymous Terms 
Software Software, project, system, application 
Effort Cost, resource, size, metric 
Estimation Estimating, estimate, prediction, predicting, predict, assessment, forecasting, forecast, calculation, 
calculate, calculating, sizing, measure, measuring 
Mobile Application Mobile software, Mobile Apps, Mobile project 
Development Improvement, Progress 
Testing Test, verification, validation 
Agile Scrum, XP, lean, crystal 
Method Process, techniques, models, approaches 





Based on the identified keywords, the search string is obtained by joining synonymous terms using 
the logical ‘OR’, other keywords using logical ‘AND’ and wildcard character (’*’). Here wildcard character 
represents 0, 1, or any number of alphanumeric characters. The following search string is obtained: 

(“Mobile Application” OR “Mobile software” OR “Mobile App” OR “Mobile project”) AND 
(“Develop*”) AND (“estimate*” OR “predict*” OR “assessment” OR “forecast*” OR “calculate*” OR 
“sizing” OR “measure*”) AND (“Improvement” OR “Progress” AND “Process” OR “techniques” OR 
“models” OR “approaches”) AND ( “agile” OR “scrum” OR ”XP” OR “lean” Or “crystal’’) 


b) Data sources 
The digital databases that are used to search the keywords are SpringerLink, IEEE Xplore, ACM 
Digital Library, Elsevier Science Direct, Wiley InterScience and Google Scholar. 


c) Search Process in Data sources 

The next phase is to apply the search string to chosen electronic data sources to find all the entailed 
studies. This phase is divided into two sub-phases: primary and secondary search phase. In Primary Search 
Phase, the electronic data sources identified are searched based on the search string defined earlier. The 
results from data sources are monitored to include search string in title and abstracts. The search string is 
again refined each time to check the outcome and analyzed for better results. The aim of this maiden check is 
to evaluate the efficacy of the search string. Additionally, results are restricted to peer-reviewed conference 
papers and journal articles which are available between 2008 and 2018 (last decade). The duplicate titles and 
abstracts are removed. In the secondary search phase, a technique called snowball tracking is used for 
studying all the references of primary studies to exploit further studies and increase the chances of inclusion 
of important papers in the systematic literature review. Table 2 lists the refined results from data sources after 
primary and secondary search phase. 


Table 2.Overview of search results 








Data Sources Relevant Search Results 
SpringerLink 3 
IEEE Xplore 75 
ACM Digital Library 57 
Elsevier Science Direct 66 
Google Scholar 83 
Wiley InterScience 20 
Total 354 





2.2.2 Inclusion/Exclusion Criteria for Selecting Studies 

The results acquired through search string defined previously in the electronic databases are 
analyzed according to the Inclusion/Exclusion criteria. The evaluation of the papers is done by reading the 
title and abstract first and checked if it is related to the issues addressed in RQs. Then the decision is made 
for its acceptance for reading the whole paper or is rejected therein. The inclusion criteria: 
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1. Studies that include estimation methods of mobile application development AND; 
2. Described in English AND; 
3. Peer-reviewed papers are selected. 
The exclusion criteria: 
1. Studies having mobile application development process and not are not considering estimation OR; 
2. Not described in English OR; 
3. Not peer reviewed 


2.2.3 Quality Assessment and Data Extraction 

To evaluate the quality of shortlisted studies; 6 questions are prepared to be answered for each 
selected study. The question can be answered as ‘Y’, ‘M ‘or ‘N’. The scores for ‘Y’ = 1, ‘P’ = 0.5, and ‘N’ = 
0. The questionnaire is developed by using the guidelines defined by Kitcheham and Charters [18]. The 
quality score for each paper should be minimum 3 to be further included in the study. 

Following are the questions in the questionnaire: 

Are the research motives clearly stated? 

Was the study designed to achieve the aims? 

Are the estimation techniques well defined? 

Is the research process documented adequately? 

Are all research questions answered adequately? 

Are the main findings stated clearly in terms of creditability, validity, and reliability? 

The authors carried out a quality assessment for the all selected studies. Four studies are excluded 
owned by their low-quality score. The data extraction phase involves extraction of data of the final selected 
studies that address the peculiarities of RQ. The data extraction for all conclusive selected studies is done in 
an MS Excel sheet containing: - Paper name, year of publication, author’ name, and paper URL. 


DE A 


3. RESULTS AND DISCUSSION 

The results of the Systematic Literature Review (SLR) and answers for Research Questions (RQs) 
are presented in this section. Table 3 describes the studies that are selected after full screening and passing 
quality assessment criteria. 


Table3. Selected papers for final studies 
StudyID ReferenceID Year  StudyID_ ReferenceID Year StudyID_ ReferenceID Year 


Sl [20] 2013 82 [21] 2015 S3 [22] 2015 
S4 [23] 2014 S5 [24] 2016 S6 [25] 2013 
S7 [26] 2014 S8 [27] 2016 S9 [28] 2017 
S10 [29] 2017 S11 [30] 2014 $12 [31] 2016 
S13 [32] 2014 S14 [33] 2008 S15 [34] 2014 
S16 [35] 2013 S17 [36] 2015 $18 [37] 2017 
$19 [38] 2017 $20 [39] 2018 $21 [40] 2016 


3.1. Traditional Techniques for Estimating Mobile Application Development: RQ1 

Sixteen studies out of twenty-one selected studies investigated the traditional estimation techniques 
for mobile applications. Table 4 lists the identified techniques where an agile approach is not followed for 
development of mobile apps. The techniques are broadly classified into three categories [20] i.e. 
Algorithmic-based models, Expert Judgment based models and analogy based models. COSMIC Function 
Size Measurement [21-28] is frequently used for estimation technique which is used to measure the 
functional size of the mobile app. Other types of estimation techniques identified are Function Point Analysis 
[29, 30] and Use Case Point [31] which is algorithmic-based models that measure functional, technical 
factors and environmental factors for estimation. Regression-Based technique [32] uses a parametric model 
based on effort predictors and data points collected through an online questionnaire which are further used in 
the regression model. Delphi method [33] is based on experience to estimate the effort whereas Architecture 
Based estimation model [34] for reliability and testing estimation of the mobile application is proposed and 
the case study was conducted in two companies. Another algorithmic approach for estimating the cost of 
developing Android mobile apps are based on the COCOMO -I and I model [35]. Analogy-based estimation 
plus functional size measurement [36] approach is also proposed for mobile apps. 
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Table 4. Traditional Estimation Techniques for mobile applications 


Estimation Techniques Study ID Approach Type 
COSMIC Function Size Measurement $1,82,S3,84,S5,S7,S11,S21 Algorithmic-based Model 
Function Point Analysis S6, S15 Algorithmic-based Model 
Delphi method S9 Expert Judgment 
Use Case Point $10 Algorithmic-based Model 
Hybrid (Analogy based estimation + Function Size $13 Analogy and Algorithmic based 
Measurement) model 
Regression-Based $12 Algorithmic-based Model 
Architecture-Based $14 Algorithmic-based Model 
COCOMO -1 and II S8 Algorithmic-based Model 





3.2. Agile techniques for estimating mobile application development: RQ2 

Agile approach to mobile application development estimation has very less number of studies. One 
of the reason could be the adoption of agile to mobile context is still in its evolving phase and many 
practitioners are adapting traditional estimation methodologies to an agile environment for mobile apps. The 
identified studies are listed in table 5. Traditional use case point method of estimation is extended by adding 
efficiency and risk factor of testers in the agile team [37]. Another technique [38] is based on a stepwise 
linear regression model which estimates the effort for Android apps from requirements specification 
including a number of use cases, actors, etc. User story point [39] is refined by considering additional factors 
along with size and complexity. The quality factor, Novelty factor and Type factor of User Story are added to 
deliver the best estimation for mobile app development. Another approach [40] uses Early Use Case Point 
(EUCP) and Extended Use Case Point (EXUCP) along with COCOMO drivers at different iteration levels in 
agile mobile app development. An experience-driven approach using the Delphi technique [41] is used for 
effort estimation having a mobile app as one of the case studies. 


Table 5. Agile Estimation Techniques for mobile applications 


Agile Estimation Techniques Study ID Approach Type 
Use Case Point S16 Algorithmic-based models 
Step-wise Linear Regression S17 Algorithmic-based models 
User story Point S18 Expert Judgment 
Use Case Pointt COCOMO $19 Algorithmic-based models 
Delphi $20 Expert Judgment 


3.3. Estimation Attributes and Accuracy Parameters for Mobile Apps: RQ3 

The estimation attributes identified in the selected studies are mostly focused on size metric whether 
based on use case, function point and story point. Table 6 lists the other estimation attributes that are 
identified for estimation. Table 7 lists the parameters used to assess the accuracy of estimation of mobile 
applications. MMRE and Pred(x) are highly followed in most of the studies. 


Table 6. Estimation attributes for mobile applications 


Estimation Attributes Study ID 
Size $1,S2,S3,84,S5.S6,S7,S10,S11,S13,S15,S16,S17,S18(user stories),S19,S21 
Cost $8,S13,S19 
Others S9(Score Metric), S12(Mean and SD of collected mobile apps variables), S14(architecture 


based), S20(mean effort based on experience) 


Table 7. Parameters for measuring the accuracy of estimation 


Accuracy Parameters Study ID 
MRE(Magnitude of Relative Error) $2,S3,S18 
MMRE(Mean Magnitude of Relative Error) $2,83,S12,S18,S19 
MdMRE(Median MRE) 82,83, $12 
Pred(percentage relative error deviation) $2,83, $12,S18,S19 
Linear Regression (R2) $12,819 
Not Defined $1,S4,S6,S7,S9S,10,S13,S14,S20,S21 
Others S8(web-based survey), S11(Compared with actual effort), $15(Compared 


with actual effort), S16(Comparison with actual effort), S17(compared with 
source code as a software measure) 
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4. CONCLUSION 

This paper represents a Systematic Literature Review (SLR) concerning effort estimation techniques 
in mobile software/applications that are developed either using traditional software development or agile 
software development methodology. Initially, the search string produced a total of 354 results and only 23 
papers are selected for primary studies. Two more papers are included after performing secondary search 
process but four papers are excluded due to low-quality scores in quality assessment criteria. The findings of 
this SLR are founded on 21 papers whose data are extracted into an excel sheet and then excavated to answer 
the RQs formed during the planning phase. 

The traditional estimation techniques applied in mobile applications are COSMIC Function Size 
Measurement, Function Point Analysis, Delphi method, Use Case Point, Hybrid(Analogy based estimation 
plus Function Size Measurement), Regression-Based, Architecture Based and COCOMO -I and I. 

The agile estimation techniques in the mobile domain are very few. The identified techniques are 
adaption or modification of traditional estimation techniques used in the agile environment to mobile 
software. The techniques are Use Case Point, Step-wise Linear Regression, User story Point with additional 
factors, Use Case Point plus COCOMO and Delphi technique. SLR also presented effort attributes and 
accuracy measures from each selected study used for assessing estimation accuracy. 

From the current studies, it can be concluded that the adoption of agile software development in 
mobile domain is proliferating from last decade [42-43] and hence there is a dire need for formal estimation 
models for mobile software. The mobile software characteristics played a critical role during estimation in 
traditional software development [4-7], [44]. It is presented in SLR that none of the estimation models for 
agile mobile application development are cogitating specific characteristics of mobile apps. This research gap 
provides ideas for researchers to devise novel models for estimation of mobile apps in agile context or extend 
existing estimation approaches of estimation. 
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